Learning from Interventions using Hierarchical Policies for Safe Learning

Learning from Demonstrations (LfD) via Behavior Cloning (BC) works well on multiple complex tasks. However, a limitation of the typical LfD approach is that it requires expert demonstrations for all scenarios, including those in which the algorithm is already well-trained. The recently proposed Learning from Interventions (LfI) overcomes this limitation by using an expert overseer. The expert overseer only intervenes when it suspects that an unsafe action is about to be taken. Although LfI significantly improves over LfD, the state-of-the-art LfI fails to account for delay caused by the expert's reaction time and only learns short-term behavior. We address these limitations by 1) interpolating the expert's interventions back in time, and 2) by splitting the policy into two hierarchical levels, one that generates sub-goals for the future and another that generates actions to reach those desired sub-goals. This sub-goal prediction forces the algorithm to learn long-term behavior while also being robust to the expert's reaction time. Our experiments show that LfI using sub-goals in a hierarchical policy framework trains faster and achieves better asymptotic performance than typical LfD.Comment: Accepted for publication at the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20

Using histogram analysis of the intrinsic brain activity mapping to identify essential tremor

BackgroundEssential tremor (ET) is one of the most common movement disorders. Histogram analysis based on brain intrinsic activity imaging is a promising way to identify ET patients from healthy controls (HCs) and further explore the spontaneous brain activity change mechanisms and build the potential diagnostic biomarker in ET patients.MethodsThe histogram features based on the Resting-state functional magnetic resonance imaging (Rs-fMRI) data were extracted from 133 ET patients and 135 well-matched HCs as the input features. Then, a two-sample t-test, the mutual information, and the least absolute shrinkage and selection operator methods were applied to reduce the feature dimensionality. Support vector machine (SVM), logistic regression (LR), random forest (RF), and k-nearest neighbor (KNN) were used to differentiate ET and HCs, and classification performance of the established models was evaluated by the mean area under the curve (AUC). Moreover, correlation analysis was carried out between the selected histogram features and clinical tremor characteristics.ResultsEach classifier achieved a good classification performance in training and testing sets. The mean accuracy and area under the curve (AUC) of SVM, LR, RF, and KNN in the testing set were 92.62%, 0.948; 92.01%, 0.942; 93.88%, 0.941; and 92.27%, 0.939, respectively. The most power-discriminative features were mainly located in the cerebello-thalamo-motor and non-motor cortical pathways. Correlation analysis showed that there were two histogram features negatively and one positively correlated with tremor severity.ConclusionOur findings demonstrated that the histogram analysis of the amplitude of low-frequency fluctuation (ALFF) images with multiple machine learning algorithms could identify ET patients from HCs and help to understand the spontaneous brain activity pathogenesis mechanisms in ET patients

Evaluating Temporal Factors in Combined Interventions of Workforce Shift and School Closure for Mitigating the Spread of Influenza

10.1371/journal.pone.0032203PLoS ONE7

Isolation and purification of lignans from Schisandra chinensis by combination of silica gel column and high-speed counter-current chromatography

Silica gel column combined with high-speed counter-current chromatography separation was successfully applied to the separation of schizandrin (I), angeloylgomisin H (II), gomisin A (III), schisantherin C (IV), deoxyschizandrin (V), γ-schisandrin (VI) and schisandrin C (VII) from the fruits of Schisandra chinensis (Turcz.) Baillon. The petroleum ether extracts of the fruits of S. chinensis were pre-separated first on a silica gel column and divided into two fractions as sample 1 and sample 2. 260 mg of sample 1 was separated by HSCCC using petroleum ether-ethyl acetate-methanol-water (10:8:10:8, v/v) as the two-phase solvent system and 18.2 mg of schizandrin, 15.7 mg of angeloylgomisin H, 16.5 mg of gomisin A and 16.7 mg of schisantherin C were obtained. 230 mg of sample 2 was separated using petroleum ether-ethyl acetate-methanol-water (10:0.5:10:1, v/v) as the two-phase solvent system and 19.7 mg of deoxyschizandrin, 23.4 mg of γ-schisandrin and 18.2 mg of schisandrin C were obtained. The purities of the separated compounds were all over 94% as determined by HPLC. The chemical structures of these compounds were confirmed by ESI-MS and 1H NMR. [Acknowledgments. Natural Science Foundation of China (20872083), scientific and technological major special project (2010ZX09401-302-5-12) and the Key Science and Technology Program of Shandong Province (BS2009SW047)

Dynamic Evolutionary Multiobjective Optimization for Raw Ore Allocation in Mineral Processing

Ding J, Yang C, Xiao Q, Chai T, Jin Y. Dynamic Evolutionary Multiobjective Optimization for Raw Ore Allocation in Mineral Processing. IEEE Transactions on Emerging Topics in Computational Intelligence. 2018:1-13.Raw ore allocation in mineral processing is crucial for improving the utilization ratio of nonrenewable raw mineral resources. Allocation of raw ore is a nonlinear, multiobjective programming problem. Such a problem is usually modeled under the assumption that the production process is stationary and the model parameters are deterministic. However, in practice, variations in equipment capability and runtime lead to frequent changes of the model parameters including the constraints. To address the above-mentioned issues, this paper formulates a raw ore allocation in mineral processing as a dynamic multiobjective optimization problem. To solve this problem, a new variant of the elitist nondominated sorting genetic algorithm (D-NSGA-II) is proposed, which employs NSGA-II as the basic component assisted by random immigrant scheme, a gradient-based local search strategy, and a mechanism for detecting environmental changes. Simulations are carried out and the results show that the proposed algorithm can efficiently achieve the Pareto front of the multiobjective raw ore allocation dynamic optimization problem. It can simultaneously converge fast while maintaining good population diversity, in particular, in the presence of large environmental changes